Interactively Learning Nonverbal Behavior for Inference and Production: A Machine Learning Approach

By designing socially intelligent robots that can more effectively communicate and interact with us, we can increase their capacity to function as collaborative partners. Our research goal is to develop robots capable of engaging in nonverbal communication, which has been argued to be at the core of social intelligence. We take a human-centric approach that closely aligns with how people are theorized to model nonverbal communication. We propose a unified computational approach to interactively learn the meaning of nonverbal behaviors for inference and production. More specifically, we use a partially observable Markov decision process model to infer an interactant’s mental state based on their observed nonverbal behaviors as well as produce appropriate nonverbal behaviors for a robot to communicate its internal state. By interactively learning the connection between nonverbal behaviors and the mental states producing them, an agent can more readily generalize to new people and situations.